Recurrence 
 Plots 
 in 


To analyze computer systems or compare them to one another, measurements are taken as the program runs. This is called a performance trace, and it can involve large amounts of time series data. Currently, statistical tools are used for analysis but this approach loses information by ignoring the ordered nature of the data. We propose improving computer performance metrics by viewing the computer as a dynamical system and studying the time varying behavior. We proposed accomplishing this by applying recurrence plots (RPs) to the data [6]. This approach provides a graphical characterization of the system that can (in some cases) filter though noise, identify non-stationary patterns, and make periodic behavior immediately apparent [7]. I tested the hypotheses that RPs will allow us to easily compare systems as well as to identify specific points in the data set that warrant further investigation. We found that RPs are not an appropriate tool for this purpose. We believe this is partially due to the fact that the scale patterns in time series data from hardware traces is much smaller than the entire time series. This difference of 1-2 orders of magnitude makes the patterns difficult to see. In addition, RPs appear too sensitive to noise to be useful for computer applications. Background The sensitivity to small perturbations is a defining characteristic of chaotic systems and is well known in computer systems (ie. irreproducibility, bugs, etc). It is exhibited within the architectural implementation (hardware) as well as code (software) and makes analysis difficult and error prone. In previous work, Bradley, Diwan, and Mytkowicz have shown that a nonlinear dynamics model of computer systems captures the effects of both factors [8]. Current work is focused on understanding the effects. This involves running and analyzing performance traces on hardware as well as simulators. This generates large amounts of time series data with significant noise. The conjecture we will explore here is that RPs will help with the data analysis by providing an additional tool for interpreting and comparing these complex systems [6]. Methods A recurrence plot (RP) is a tool which allows recurrence patterns in time series data to be visualized in two dimensions [9]. This is done by plotting points where the trajectory at time i is close to the trajectory at time j. This can produce a colored graph corresponding to a range of differences (unthresholded), or a black and white graph (thresholded) where only the points outside a specified threshold are plotted. Mathematically, a thresholded RP corresponds to Ri,j = 1 if: |xi − xj | < τ else Ri,j = 0, where R is the matrix corresponding to the recurrence plot, xt corresponds to displacement at time t, and τ is some threshold value. An RP always contains the line y = x about which it is symmetric (reflecting the fact that the system’s state is equal to itself at every point in time). Lines parallel to this diagonal signify recurrences. This is useful in identifying periodicities, limit cycles, or chaotic behavior. Recurrence plots allow one to look beyond noise in the system as well as to identify non-stationary patterns and other interesting points [3, 1]. The data to be studied here consists of example data generated for the purpose of understanding the tools as well as instructions per cycle (IPC) data taken from a simulator. They both consist of some measurement as a function of time. We also analyzed cache misses (a memory usage metrics) but these results are not included in the report. Data was gathered by other members of the team (specifically Todd and Stephen) from simulators as well as real hardware systems. Measurements were taken every hundred thousand cycles and later normalized to ”per cycle” count to produce IPC. When calibrating the RP software, I down sampled the data in order to achieve a manageable size. The goal of this work was to evaluate RPs as a tool for identifying patterns and points of interest in hardware traces. The data from such traces is long and difficult to interpret. We explored whether RPs aided in the analysis of these data. PART 2: Understand and Calibrate Tools In order to evaluate the effectiveness of recurrence plots in identifying points of interest in large time series data, I first looked at data with known characteristics. This served the purpose of both developing my understanding of the tools as well as determining the strengths and weaknesses of RP analysis. I chose to look at the effects of two characteristics that are likely in our experimental data: noise and drift. Noise Experiment: Noise is present in virtually every experimental data set. In our hardware traces (from real systems), many steps have been taken to limit the amount of noise, but it is impossible to completely eliminate it. For example, we can ensure that there are no superfluous programs running when we are taking our measurements but we cannot turn off the operating system. Therefore, our data contains known noise from the operating system interrupting and using cycles of the CPU to perform tasks unrelated to our test programs. It is highly likely that there exists noise from a variety of unknown factors. In an effort to quantify effects of noise on the analysis, I generated example data with varying amounts of noise. The data was generated from the following function: